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ARl  Research  Reports  and  Technical  Reports  are  intended  for  sponsors  of 
R&D  tasks  and  for  other  research  and  military  agencies.  Any  findings  ready 
for  implementation  at  the  time  of  publication  are  presented  in  the  last  part 
of  the  Brief.  Upon  completion  of  a  major  phase  of  the  ta.sk,  formal  recom¬ 
mendations  for  official  action  normally  are  conveyed  to  appropriate  military 
agencies  by  briefing  or  Disposition  Form. 


FOREWORD 


The  Instructional  Technology  Systems  Technical  Area  of  the  US  Army  Research 
Institute  for  the  Behavioral  and  Social  Sciences  performs  research  and  develop¬ 
ment  in  educational  technology  with  applications  to  military  education  and 
training.  A  major  focus  of  this  research  is  the  development  of  information  on 
which  the  Department  of  the  Army  can  base  decisions  about  its  Basic  Skills 
Education  Program.  This  report  compares  three  segments  of  the  Basic  Skills 
Program — six-week,  three-month,  and  six-month  English-as-a-Second  Language 
programs  that  were  provided  to  eligible  soldiers  prior  to  their  entry  into 
Basic  Combat  Training. 

Soldiers  enrolled  in  all  three  programs  demonstrated  gains  in  scores  on 
the  English  Comprehension  Level  Test  at  the  same  orderly,  predictable  rate. 

The  longer  the  training  time,  the  greater  the  gains.  Gains  in  scores  on  the 
standardized  test  were  substantially  higher  for  all  three  groups  of  enrolled 
soldiers  than  they  were  for  a  group  of  eligible  soldiers  who  went  directly 
into  Basic  Training  and  were  retested  six  to  nine  months  later.  Also,  higher 
test  scores  correlated  with  lower  attrition  during  training.  The  Army  must 
consider,  therefore,  whether  to  assign  soldiers  to  English  language  programs 
of  varying  durations  on  the  basis  of  differing  deficiency  levels. 

This  research  effort  was  supported  by  the  Office  of  the  Adjutant  General 
and  the  Training  and  Doctrine  Command. 


EDGAR  M.  JOHNSON 
Technical  Director 
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EXECUTIVE  SUMMARY 


Requirement : 

To  evaluate  the  effectiveness  of  Army  English-as-a-second 
language  (ESL)  programs  of  various  lengths  and  types. 


Procedure : 

Three  ESL  programs--six-week ,  three-month,  and  six-month-- 
were  evaluated  with  a  standardized  group  test  of  English  com¬ 
prehension,  an  individually  administered  test  of  oral  proficiency, 
questionnaires,  observations,  followup  questionnaires  and  tests, 
informal  techniques,  and  curriculum  analysis.  Procedures  were 
designed  with  the  support  of  The  Adjutant  General’s  Office  and  the 
cooperation  of  the  Training  and  Doctrine  Command. 


Findings : 

Most  of  the  soldiers  in  ESL  programs  were  well  educated 
Puerto  Ricans  whose  English  speaking  skills  were  weak  The  three 
programs  differed  considerably  in  their  length  and  content, 
particularly  their  degree  of  ’’functionality."  All  three  programs 
produced  gains  in  English  proficiency  as  measured  by  the  standard¬ 
ized  test.  The  longer  the  training  time,  the  greater  the  gains. 
Oral  proficiency  data  showed  that  six-week  students  gained  at 
about  the  same  rate  as  three-month  students  in  some  skills  but  not 
in  others.  ESL  was  found  to  have  some  positive  relationship  to 
lowering  of  attrition  rates.  Although  ESL  participants  generally 
liked  their  programs  and  teachers,  they  felt  the  need  for  more 
practice  in  conversation  skills.  Supervisors  were  generally 
supportive  of  the  programs  and  made  qualitative  distinctions 
between  soldiers  scoring  50  or  more  on  the  standardized  test  and 
those  scoring  less  than  50. 


Utilization  of  Findings: 

The  findings  have  relevance  for  scientists  and  administrators 
in  military  education  and  training. and  for  all  personnel  who  deal 
with  limited  English  proficient  soldiers.  Implications  of  the 
results  concern  key  characteristics  of  ESL  programs,  the  need  for 
ESL  in  the  Army,  and  the  validity  of  typicially  used  cutoff  scores 
on  the  standardized  test  of  English  comprehension. 
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ENGLISH-AS-A-SECOND-LANGUAGE  PROGRAMS  IN  THE  ARMY 


INTRODUCTION 

This  report  presents  results  of  an  evaluation  of  English-as-a- 
second-language  (ESL)  training  programs  in  the  Army.  These  pro¬ 
grams  are  part  of  the  Army’s  Basic  Skills  Education  Program  (BSEP). 
ESL  instruction  is  offered  to  soldiers  who  need  greater  English 
proficiency  to  function  well  in  the  Army.  The  focus  of  this  report 
is  ESL  as  taught  before  or  during  basic  training  (BT).  Results  are 
reported  for  three  programs,  the  standard,  six-week  ESL  program 
given  in  1982  at  six  training  bases  and  two  one-time,  experimental 
programs--one  a  three-month  and  the  other  a  six-month 
program--conducted  for  the  Army  in  1981-1982  at  the  Defense  Lan¬ 
guage  Institute  English  Language  Center  (DLIELC)  at  Lackland  Air 
Force  Base,  Texas. 

This  evaluation  is  important  because  it  concerns  ESL,  which 
becomes  increasingly  more  critical  as  the  number  of  limited  English 
proficient  persons  in  the  United  States  grows.  It  is  an  example  of 
multitrait-multimethod  research  conducted  in  difficult,  ’’real 
world”  settings.  It  also  reflects  the  joint  efforts  of  several 
agencies,  namely  the  U.S  Army  Research  Institute  for  the  Be¬ 
havioral  and  Social  Sciences  (ARI),  which  conducted  the  research 
through  contract  to  the  American  Institutes  for  Research  (AIR);  the 
Office  of  the  Adjutant  General,  which  supported  the  research;  and 
the  Training  andDoctrine  Command  (TRADOC) ,  which  provided  essen¬ 
tial  cooperation  in  many  areas  of  the  research. 


The  Problem 

The  Army  estimates  that  at  least  5%  of  the  present  enlisted 
force  has  English  language  comprehension  problems.  We  can  expect 
this  percentage  to  increase  over  the'next  decades  as  the  armed 
services  induct  increasing  numbers  of  limited  English  speakers. 
Recruits  who  have  trouble  understanding  and  speaking  English  find 
it  difficult  to  complete  training  successfully.  In  the  training 
environment  language  difficulties  are  associated  with  many  prob¬ 
lems:  culture  shock,  difficulty  in  following  directions  and 
understanding  written  material,  low  motivation,  and  safety  prob¬ 
lems,  according  to  TRADOC  (1982).  Similarly,  a  recent  study  by  the 
Navy  found  that  1 imited-English-speaking  Hispanic  recruits  had 
higher  attrition  rates,  reduced  promotion  potential,  and  decreased 
job  efficiency  compared  to  English-speaking  recruits  (Salas, 
Kincaid,  &  Ashcroft,  1980).'  These  correlates  of  limited  English 
proficiency  have  high  costs  for  the  military.  It  is  necessary  to 
find  ways  to  train  limited  English  proficient  recruits  so  that  they 
can  perform  well  in  the  armed  services,  and  this  training  will 
likely  involve  some  form  of  ESL. 


ESL  Approaches;  Traditional  and  Modern 


Traditionally,  foreign  or  second  languages  were  taught  as 
grammar  and  translation  courses  with  more  emphasis  on  reading  than 
on  communicating.  However,  the  needs  of  the  military  and  the 
foreign  service  to  produce  fluent  speakers  of  foreign  languages  led 
to  the  development  of  intensive  courses  that  emphasized  oral  rather 
than  reading  skills.  The  methods  developed  in  these  contexts 
filtered  back  to  high  schools  and  into  adult  language  programs. 
These  methods  ware  based  on  the  common-sense  principle  that,  in 
order  to  speak  a  language  well,  one  has  to  practice  speaking  it 
(Anthony  8e  Norris,  1972). 

Drawing  on  the  principles  of  behavioral  psychology,  developers 
of  foreign  language  and  ESL  courses  created  the  audio-lingual  or' 
aural-oral  method.  In  a  purely  structural  audio-lingual  ESL 
course,  trainees  learn  different  sentence  structures  through 
pattern  drills.  The  purely  structural  approach  is  limited  because 
of  difficulties  in  transfer  of  training  from  the  classroom  to  the 
"real  world"  (Crandall,  1979). 

Purely  structural  ESL  gave  way  to  more  situational  or  "func¬ 
tional"  ESL,  in  which  structures  are  taught  and  practiced  in  the 
context  of  dialogues  reflecting  situations  the  trainees  might 
normally  encounter  outside  the  classroom.  Making  the  situations 
relevant  addresses  not  just  transfer  of  training  but  also  motiva¬ 
tion  (Knowles,  1978;  Schumann,  1978).  Moreover,  functional  situa¬ 
tions  provide  the  opportunity  to  teach  culture  as  well  as  language. 
A  relatively  new  development  is  the  movement  to  a  more  generalized 
functional  curriculum  organized  around  speech  acts  (e.g.,  making 
requests,  asking  for  clarification,  expressing  an  opinion)  rather 
than  based  on  situations.  This  is  known  as  a  "notational- 
functional"  curriculum  (Crandall,  1979). 

Carroll's  (1967)  work  on  foreign  language  learning  among 
college  students  is  relevant  for  all  types  of  ESL  approaches. 
Carroll  found  that  the  major  variable  influencing  foreign  language 
competence  was  the  amount  of  time  the  student  was  involved  in  using 
the  language.  Research  on  "time  on  task"  (Denham  &  Lieberman, 

1980)  substantiates  the  importance  of  the  time  factor  for  academic 
progress. 

Until  1982  the  Army's  ESL  training  was  mainly  structural.  In 
1982  a  more  functional.  Army-related  ESL  course  was  initiated,  but 
the  Army  has  not  yet  moved  to  functional-notational  ESL.  Different 
lengths  of  programs  have  been  piloted  by  the  Army  with  an  under¬ 
standing  that  the  time  variable  might  be  important.  The  current 
research  concerns  three  of  the  Army's  structurally  oriented  ESL 
programs  of  varying  durations:  six  weeks,  three  months,  and  six 
months.  Data  will  be  collected  later  on  the  functional  ESL  pro¬ 
gram  . 
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Evaluation  Questions 


In  the  context  of  the  problem  and  the  ESL  approaches  described 
above,  the  investigation  focused  on  five  key  questions: 

Question  1;  What  were  the  characteristics  of  students  -in 
the  ESL  programs? 

Question  2:  What  were  the  characteristics  of  the  three  ESL 
programs? 

Question  Did  any  of  the  three  programs  improve  soldiers' 
ability  to. use  English? 

Question  4:  Did  any  of  the  three  programs  result  in  lower 
attrition  rates  for  soldiers? 

Question  5:  What  were  the  perceptions  of  ESL  students  and 
their  supervisors  toward  ESL  programs? 


METHOD 

This  section  discusses  subject,  data  gathering  procedures,  and 
statistical  analysis. 


Subjects 

The  subject  pool  consisted  of  participants  in  three  ESL 
programs,  including  a  six-week  regular  program,  a  three-month 
experimental  program,  and  a  six-month  experimental  program  (see 
Table  1).  The  six-week  regular  program  during  fiscal  years  1979 
through  1981  (FY79~81)  involved  2,804  participants,  for  2,232  of 
whom  standardized  test  data  are  available.  A  nonmatched  comparison 
group  consisting  of  1,679  eligible  nonenrollees  was  used  for  some 
analyses.  The  three-month  experimental  program  involved  151 
soldiers  (of  whom  148  graduated)  -and  had  a  nonmatched  comparison 
group  of  99  soldiers.  Comparison  group  data  for  the  three-month 
experiment  are  not  reported  here  because  they  are  still  being 
gathered.  A  total  of  200  participants  (of  whom  186  graduated)  and 
200  controls  matched  on  ESL  eligibility  were  part  of  the  six-month 
experiment.  Some  of  the  control  group  members  received  no  English 
training  in  the  Army,  while  others  attended  the  six-week  regular 
course.  The  control  group  was  pretested  at  Army  entry  and 
posttested  nine  months  later.  Optimally,  posttesting  would  have 
been  six  months  later,  but  Army  operational  schedules  precluded 
this. 


Table  1 

Number  of  Subjects  from  Three  ESL  Programs 
Participa.'ting  in  Individual  Da1;a  Collocliion  Pro.cedurea 
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Data  Gathering  Procedures 


The  three  programs  were  evaluated  with  a  standardized  group 
test  of  English  comprehension,  an  individually  administered  test  of 
oral  proficiency,  questionnaires,  observations,  followup  question¬ 
naires  and  tests,  informal  techniques,  and  curriculum  analysis. 

Not  all  these  methods  were  .used  for  every  program,  in  part  because 
the  six-month  program  had  been  completed  before  field  data  collec¬ 
tion  began  and  in  part  because  the  large  data  base  is‘  strictly 
archiv  al . 

Standard  Group  Test  of  English  Comprehension 

All  three  programs  were  evaluated  through  the  use  of  the 
English  Comprehension  Level  Test  (ECLT).  The  ECLT  was  administered 
before  and  after  the  experimental  programs,  with  interim  testing 
approximately  every  two  weeks  during  the  programs.  For  the  six- 
week  program,  the  ECLT  was  given  before  and  after  training  and,  at 
some  installations,  as  a  midterm  exam.  The  ECLT  was  developed  by 
DLIELC.  Forms  are  replaced  annually.  Each  form  has  120  items. 

All  items  are  multiple-choice  with  four  possible  answers  per  item. 
Respondents  mark  answers  on  a  separate  answer  sheet.  Lexical  items 
comprise  75?  of  the  test  content,  and  the  remainder  of  the  test 
contains  structural  items.  Two-thirds  of  the  test  are  devoted  to  a 
combination  of  listening  and  reading,  and  the  balance  is  purely 
reading  (DLIELC,  1980). 

The  ECLT  has  high  reliability  and  validity.  Internal  consis¬ 
tency  and  alternate  forms  reliability  are  in  the  .80-. 90  range 
(DLIELC,  1980).  When  correlated  with  adult  basic  education  tests 
and  tests  of  English  as  a  foreign  language,  the  ECLT  has  concurrent 
validity  coefficients  in  the  .79-. 83  range  (DLIELC,  1980;  Mebane, 
1980;  Oxford-Carpenter,  Leopold,  &  Duvall,  1982).  Although  the 
perils  of  language  testing  have  been  clearly  noted  (Troike,  1983), 
the  ECLT  appears  to  have  both  psychometric  merit  and  operational 
utility. 

Individually  Administered  Test  of  Oral  Proficiency 

About  one-third  (N=M3)  of  t1>e  three  month  group  and  1.4? 

(N=33)  of  the  six-week  group  were  chosen  to  take  a  newly  developed, 
individually  administered  test  of  oral  language  proficiency  at  the 
beginning  and  the  end  of  their  ESL  training.  Three-month  par¬ 
ticipants  were  randomly  selected  within  each  of  the  three  ECLT 
levels  to  take  the  oral  test.  Soldiers  sampled  in  the  six-week 
program  were  those  who  entered  the  program  at  a  given  installation 
during  the  w«»ek  of  the  evaluator's  first  visit;  they  received  the 
oral  test  as  a  posttest  five  weeks  later. 
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The  oral  test  was  developed  by  the  American  Institutes  for 
Research  (AIR)  to  determine  an  individual  soldier's  skill  levels  in 
English  comprehension  and  speaking.  The  test  includes  overall 
scores  for  production  and  comprehension  and  subscores  for  fluency, 
pronunciation,  grammar,  vocabulary,  and  specific  comprehension. 

The  test  was  built  on  the  Foreign  Service  Institute  model  but 
includes  content  relevant  to  military  life  and  involves  some 
pictorial  cues.  Preliminary  concurrent  validity  data  show  that  the 
subtests  correlate  .70-. 89  with  the  ECLT.  Furthermore,  pre-post 
gains  on  the  oral  test  correlate  with  ECLT  gains  in  the  range  of 
.50-. 71  for  all  subtests  except  pronunciation  and  total  comprehen¬ 
sion.  Interrater  reliability  coefficients  for  the  subtests  were 
.75-. 99  for  the  six-week  program  sample  and  .80-. 99  for  the  three- 
month  program  sample. 

Questionnaires 


Questionnaires  about  language  proficiency,  training,  back¬ 
ground,  and  attitude  were  given  to  soldiers  in  the  six-week  and 
three-month  programs.  Spanish  speakers  received  questionnaires  in 
Spanish.  BT  drill  sergeants  and  Advanced  Individual  Training  (AIT) 
instructors  received  questionnaires  concerning  perceptions  about 
soldiers  who  had  participated  in  the  three-  and  six-month  programs. 

Observations 


For  the  three-month  and  six-week  courses,  observations  were 
made  of  instructional  settings  and  techniques  using  a  standardized 
form.  In  many  instances  it  was  possible  to  verify  observational 
data  through  multiple  visits  to  posts.  Although  observational  data 
were  not  available  for  the  six-month  program,  retrospective  anecdo¬ 
tal  data  obtained  through  instructor  interviews  were  available. 

Followup 


Each  program  had  some  kind  of  followup.  Participants  in  the 
six-week  program  in  FY79-81  were  tracked  by  records  on  attrition 
and  other  variables.  A  followup  of  six-month  program  participants 
involved  gathering  data  from  BT  and  AIT  supervisors  on  soldiers' 
attitudes,  language  performance,  and  training  base  adjustment.  An 
even  more  elaborate  followup  was  conducted  on  three-month  program 
participants. 

informal  Interviews  and  Observations 

Informal  methods,  such  as  unstructured  interviews  and  field 
notes,  were  used  to  supplement  formal  techniques.  Unstructured 
Spanish-language  interviews  with  program  participants  proved  to  be 
particularly  important. 


Curriculum  Analysis 


Curriculum  elements,  including  structure,  texts,  workbooks, 
tests,  and  other  materials,  were  analyzed  carefully  for  all  three 
programs. 


Statistical  Analysis 

Because  most  of  the  analyses  concerned  the  total  population 
rather  than  a  sample,  and  because  those  samples  which  were  used 
were  not  randomly  assigned,  descriptive  rather  than  inferential 
statistics  were  more  suitable. 


RESULTS 

The  results  are  presented  in  the  order  of  the  five  questions 
listed  earlier. 


Question  1;  VJhat  Were  the  Characteristics  of  the  Students 

in  the  ESL  Programs'^ 

During  FY79-81,  at  least  4,^83  limited  English  speaking 
soldiers  were  identified  as  eligible  for  ESL  instruction,  with 
eligibility  based  on  a  score  of  less  than  70  on  the  ECLT  and/or  on 
referral  by  commanders.  ESL  instruction  is  officially  required  for 
eligible  Regular  Army  enlistees,  although  only  about  two-thirds 
actually  enroll.  ESL  is  optional  for  members  of  the  National  Guard 
and  the  Enlisted  Reserves.  These  figures  come  from  the  TRADOC  data 
base  and  are  underestimates,  since  data  are  known  to  be  missing. 
Extrapolating  from  the  data  base  available  during  surveys  of  ESL 
classes  in  FY82,  we  estimate  that  the  enrollment  for  FY82  alone  was 
1,500-2,000  soldiers. 

Between  85%  and  95%  of  the  Army's  ESL  students  are  Spanish 
speakers,  and  most  of  these  hail -from  Puerto  Rico.  Most  of  the 
non-Spanish  speakers  are  from  Korea  and  the  Philippines.  Puerto 
Rican  ESL  soldiers,  by  far  the  vast  majority,  are  almost  all  high 
school  graduates  who  are  literate  in  their  native  language.  Some 
have  college  experience.  Most  have  studied  English  in  a  grammar- 
translation  mode  in  school.  The  typical  ESL  soldier  has  some 
facility  in  reading  and  writing  English  but  very  weak  skills  in 
speaking  English  and  understanding  spoken  English.  Despite  their 
previous  language  training,  ’s.tudents'  scores  on  the  ECLT  are  widely 
distributed  over  the  ESL-eligible  range  (0-70). 


Question  2:  What  Were  the  Characteristics  of  the  Three 

ESL  Programs? 


In  FY82,  six  continental  United  States  installations — Forts 
Banning,  Dix,  Jackson,  Knox,  Leonard  Wood,  and  Sill — had  regular 
six-week  ESL  programs  for  new  recruits.  Forts  Gordon  and  Bliss  had 
operated  large  programs  during  FYTQ-SI,  but  in  1982  they  did  not 
house  BT  units  and  therefore  did  not  have  ESL. programs .  The  ESL 
programs  varied  in  size,  ranging  from  Fort  Leonard  Wood  with  one  or 
two  teachers  and  an  average  of  one  soldier  entering  ESL  per  week  to 
Forts  Jackson  and  Dix  with  six  or  seven  teachers  and  an  average  of 
seven  to  12  soldiers  entering  ESL  per  week.  The  six-week  programs 
also  varied  in  the  emphasis  they  placed  on  teaching  English  lan¬ 
guage  structure  versus  military  information.  Al-though  the  cur¬ 
riculum  of  .the  six-week  programs  was  supposed  to  be  the  American' 
Language  Course  (ALC)  developed  by  DLIELC,  over  time  most  installa¬ 
tions  replaced  or  supplemented  the  ALC  with  their  own  lessons  on 
military  information  and  with  other  teacher-made  materials.  The 
ALC  is  a  standardized,  multimodal,  oral-aural  program  which  focuses 
mainly  on  listening,  reading,  and  pattern  practice.  Soldiers  in 
the  six-week  programs  attended  class  six  hours  a  day,  five  days  a 
week.  At  some  installations  ESL  students  were  housed  as  a  group 
with  other  basic  skills  trainees  in  one  unit.  They  were  then 
assigned  to  separate  units  for  BT  after  completing  the  ESL  course. 
At  other  installations  ESL  and  other  basic  skills  trainees  were 
housed  with  soldiers  who  were  already  in  BT;  they  remained  in  those 
units  until  completion  of  ESL  or  BT. 

The  first  of  the  two  DLIELC  experimental  programs  lasted  26 
weeks,  or  approximately  six  months,  September  1980-March  1981.  The 
second  experimental  program  lasted  13  weeks,  or  approximately  three 
months,  August-November  1981.  For  the  six-month  program,  ESL 
training  was  considered  part  of  each  recruit’s  enlistment  obliga¬ 
tion.  Students  were  selected  from  a  pool  of  eligible  recruits. 
Eligible  soldiers  who  went  directly  to  the  training  base  served  as 
matched  controls.  Soldiers  in  the  three-month  program  had  to  agree 
to  extend  their  service  obligation  by  three  months.  Both  ex¬ 
perimental  orograms  were  conducted  by  regular  DLIELC  teachers  using 
the  ALC.  Soldiers  in  these  two  programs  were  housed  as  Army  troops 
under  supervision  of  Army  noncommissioned  officers  (NCOs) ,  away 
from  Air  Force  personnel  and  foreign  nationals  at  Lackland  Air 
Force  Base.  After  ESL  classes  soldiers  were  treated  as  a  military 
unit  and  exposed  to  military  topics  by  their  drill  sergeants. 
Soldiers  attended  ESL  classes  six  hours  per  day  (two  hours  of  which 
were  in  a  language  1 aboratory ) , . f ive  days  per  week. 
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Question  3:  Did  Any  of  the  Three  Programs  Improve  Soldiers* 

Ability  to  Use  English? 

All  three  programs  produced  gains  in  English  proficiency.  Table 
2  shows  average  weekly  ECLT  gains  and  average  total  ECLT  gains  for 
the  three  programs  and  for  the  six-month  control  group.  As  shown, 
participants  in  all  three  programs  made  substantially  higher 
average  weekly  gains  (ranging  from  1.3  to  1.9  points)  than  the 
control  group  (.5  points).  The  variation  in  gain  for- the  three 
programs  was  not  great,  with  participants  in  the  three-month  and 
six-week  programs  achieving  approximately  equal  average  weekly 
gains  (1.9  and  1.7  points)  and  participants  in  the  six-month 
program  gaining  1.3  points  per  week  on  the  average.  Average  weekly 
gain  is  a  convenient  and  useful  metric  for  comparing  programs. 
However,  use  of  average  weekly  gain  does  not  imply  or  confirm 
linearity  of  gain  scores  across  time.  It  is  important  to  note  that 
a  ceiling  effect  may  have  occurred  for  six-month  program  par¬ 
ticipants,  thus  suppressing  their  average  weekly  gain  below  that  of 
participants  in  the  other  two  programs. 

The  average  total  gain  for  the  programs  indicated,  as  would  be 
expected  based  on  Carroll's  (1967)  study,  that  the  longer  a  soldier 
has  ESL  training,  the  greater  the  English  proficiency  gains.  The 
average  total  gain  for  the  six-month  program  was  31  points;  for  the 
three-month  program,  23  points;  for  the  six-week  program,  10.2 
points;  and  for  the  six-month  control  group,  18  points. 

Figures  1  and  2  show  that  the  three-  and  six-month  experimen¬ 
tal  programs  had -similar  learning  curves  for  different  levels  of 
ECLT  pretest  scores.  In  other  words,  average  rates  of  improvement 
in  English  proficiency  were  quite  regular  for  both  programs.  Gains 
were  made  by  participants  in  all  entry  ability  levels. 

Results  of  the  oral  proficiency  test  (given  to  six-week  and 
three-month  participants)  also  showed  gains  in  English  proficiency. 
As  shown  in  Table  3>  the  six-week  students  gained  on  all  measures 
except  pronunciation,  while  the  three-month  students  gained  on 
every  measure.  In  overall  comprehension,  grammar,  and  fluency,  the 
six-week  stucients  gained  at  about  the  same  rate  as  the  three-month 
students.  On  the  other  parts  of  -the  test,  the  six-week  students 
gained  at  slower  rates  than  the  three-month  students.  Most  six- 
week  total  gains  were  24%  to  50%  of  the  magnitude  of  three-month 
total  gains.  Ignoring  differences  in  curriculum,  a  simple  time 
comparison  would  lead  to  the  expectation  that  the  total  gain  for 
the  six-week  program  would  be  50%  of  that  of  the  three-month 
program,  since  the  six-week  program  was  half  as  long  as  the  three- 
month  program.  At  this  point  we  do  not  know  if  a  different  type  of 
program  would  have  produced ’greater  gains  in  oral  proficiency  in 
six  weeks. 
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Table  2 


Average  ECLT  Gains  by  Program^- 


Program 

N 

•  Points/week 

Total  points  gained 

Six  months  (26  weeks) 

185 

1-3 

31 

Three  months  (15  weeks) 

151 

1.9 

25 

Six  weeks^ 

2,252 

1.7 

10.2 

Six-month  control^ 

50 

.5 

1-8 

^•Note  that  these  figures  are  not  to  be  construed  as  suggesting  that 
all  gains  were  linear.  Use  of  average  weekly  gain  simply  provides 
a  comparison  of  programs  but  does  not  confirm  linearity. 

“Counting  only  soldiers  completing  five  or  more  weeks.  (Counting 
all  2,804  cases  in  the  TRADOC  base  gives  an  average  gain  of  2 
points/week.  Soldiers  who  exit  early,  however,  make  unprecedented 
high  gains  in  a  very  short  time.  These  gains  are  probably  spurious 
Early  exit  was  not  an  option  in  the  DLIELC  courses,  although  some 
soldiers  reached  the  maximum  score  (80+)  before  the  end  of  the 
course.  In  calculating  mean  gains  for  the  TRADOC  data,  we  set  all 
scores  below  50  to  20  to  make  them  comparable  with  the  DLIELC  data. 
^Over  a  nine-month  period. 


'Table  5 


Oral  Proficiency  Gains  for  fne  T'nree-Month^-  Experimental 
Program  Participants  and  the  Six-Week  Regular  Program. Participants 


Oral 

proficiency 

measure 


Three-month 

gains 


Six-week 

gains 


^13  weeks  of  instruction. 


r 


Table  4 

AIT  Failures®-  by  ECLT  Entry  Scores  for  Total  Six-Month  Experimental 
Program  Participants  and  Control  Group  Members 


p 


;V 


For  both  groups  that  took  the  oral  proficiency  test,  overall 
comprehension  improved  faster  than, overall  production.  This  is 
consistent  with  child  development  literature,  which  shows  that 
production  may  demand  more  practice  than  comprehension  and  may 
depend  on  comprehension. 

In  interpreting  these  proficiency  data,  it  is  important  to 
remember  that  most  students  in  Army  ESL  programs  were  Puerto  Ricans 
who  had  previously  studied  English  in  school  but  lacked  English 
speaking  skills.  Hence,  Army  ESL  training  may  be  considered 
retraining  for  many  or  most  of  the  participants. 


Question  Did  Any  of  the  Three  Programs  Result 
in  Lower  Attrition  Rates  for  Soldiers? 

The  general  answer  to  this  question  is  that  ESL  training 
helped  participants,  especially  lower~scoring  ones,  to  stay  in  the 
Army.  Attrition  is  defined  here  as  failing  AIT  or  leaving  anytime 
before  completing  AIT.  Findings  for  the  six-month  program  are 
shown  in  Table  M.  When  six-month  participants  were  compared  with 
the  matched  control  group,  attrition  was  far  greater  for  the 
control  group  (9.9^)  than  for  the  experimental  group  (0.5?). 

Similar  comparative  data  are  not  yet  available  for  the  three-month 
program.  For  the  six-week  program.  Table  5  indicates  that  the 
percentage  of  soldiers  who  failed  or  left  before  completing  AIT  was 
about  the  same  for  participants  and  eligible  non-participants. 
Furthermore,  soldiers  who  entered  the  six-week  program  with  higher 
ECLT  scores  enjoyed  no  advantage  over  those  with  lower  entry 
scores.  According  to  these  findings,  then,  the  six-week  program 
had  no  apparent  effect  on  training  base  attrition;  yet  other  data 
(shown  in  Table  6)  provide  some  evidence  of  an  influence  of  the 
six-week  program  on  attrition.  Table  6  indicates  that  higher  ECLT 
gains  are  associated  with  lower  attrition  for  the  six-week  program 
participants,  and  lower  gains  are  linked  to  higher  attrition.  In 
sum,  ESL  participation  clearly  affected  attrition  rates  for  six- 
for  three-month  program  students. 


Question  5:  What  Were  the  Perceptions  of  ESL  Students  and 
Their  Supervisors  Toward  ESL  Programs? 

There  were  mixed  but  generally  positive  perceptions  about  the 
three  programs  among  students  and  their  BT  and  AIT  supervisors. 
Highlights  of  the  questionnaire  results  are  presented  below. 


s' 
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Perceptions  About  the  Six-Week  Regular  Program 


More  than  90%  of  the  participants  said  that  their  English  had 
improved  at  least  a  little  since  they  joined  the  Army,  at  which 
time  they  began  their  Army  ESL  training.  More  than  90%  said  their 
teachers  care  about  their  learning  English  and  try  to  help  them, 
and  80%  said  they  like  the  way  their  teachers  teach  ESL,  Most  stu¬ 
dents  (76%)  expressed  good  feelings  about  the  overall  ESL  program. 
Almost  all  students  (95%)  who  completed  the  course  expressed 
greater  communicative  competence  than  they  felt  when  they  entered 
the  Army.  However,  almost  all  students  (92%)  also  reported  the 
need  to  improve  speaking  and  understanding  English.  They  perceived 
speaking  to  be  their  greatest  weakness  and  felt  that  they  received 
too  little  training  in  speaking  and  listening.  -Even  after  five  to 
six  weeks  of  training,  nearly  half  the  students  rated  themselves ' as 
"poor"  or  "not  at  all"  in  speaking  English,  and  two-fifths  rated 
themselves  similarly  in  understanding  spoken  English.  Eighty-five 
percent  said  they  needed  more  chances  to  practice  speaking  English 
in  class,  and  nearly  90%  said  they  did  not  get  enough  opportunities 
to  talk  with  English  speakers.  Respondents  attributed  their 
English  improvement  more  to  encounters  with  English  speakers 
outside  of  class  than  to  classroom  activities,  particularly  at 
posts  where  ESL  students  were  billeted  with  English  speakers. 

Perceptions  About  the  Three-Month  Experimental  Program 

As  in  the  six-week  program,  most  soldiers  who  participated  in 
the  three-month  program  wanted  more  instruction  in  conversation 
skills  and  less  instruction  in  reading  English.  Understanding 
spoken  English  was  the  highest  need  expressed  by  these  soldiers 
when  followed  up  in  BT  (56%),  while  speaking  English  was  the 
highest  need  they  mentioned  in  AIT  (86%).  In  both  BT  and  AIT, 
supervisors'  ratings  closely  reflected  exit  ECLT  scores  of  sol¬ 
diers.  Soldiers  scoring  50  and  above  on  the  exit  ECLT  tended  to 
receive  higher  ratings  by  BT  and  AIT  supervisors  for  understanding, 
speaking,  reading,  and  writing  English  than  did  soldiers  who  scored 
less  than  50  on  the  exit  ECLT.  BT  and  AIT  supervisors  rated  ESL 
soldiers  higher  in  speaking  and  understanding  English  than  in 
reading  and  writing  English,  This  may  be  explained  by  the  fact 
that  that  these  supervisors  did  not  have  much  opportunity  to 
observe  soldiers'  reading  and  writing  skills.  In  contrast  to  their 
supervisors,  ESL  soldiers  rated  themselves  as  least  competent  in 
speaking  English  in  both  BT  and  AIT.  In  BT  they  felt  themselves 
most  competent  in  reading  English,  and  in  AIT  they  said  they  were 
most  skilled  in  understanding  classroom  lectures.  Supervisors  gave 
high  ratings  to  the  motivation  of  ESL  participants. 

Perceptions  About  the  Six-Month  Experimental  Program 

Six-month  program  graduates  who  had  exit  ECLT  scores  below  50 
were  far  more  likely  to  be  rated  by  AIT  instructors  as  "extremely 
limited"  in  English  speaking  and  understanding  than  were  graduates 
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with  50+  exit  ECLT  scores.  Specifically,  23J  of  the  below-50 
graduates  were  considered  extremely  limited  in  understanding,  and 
1A%  of  the  below-50  were  considered  extremely  limited  in  speaking, 
compared  to  and  2%  of  the  50+  graduates.  BT  drill  sergeants' 
ratings  were  in  the  same  direction,  but  the  difference  between 
below-50  and  50+  was  less  pronounced. 


Summary  of  Results  Across  Questions 

Most  of  the  soldiers  in  ESL  programs  were  well  educated  Puerto 
Ricans  whose  English  speaking  skills  were  weak.  Programs  differed 
considerably  in  their  length  and  content  (particularly  in  their 
degree  of  "functionality").  All  three  programs  that  were  inves¬ 
tigated  produced  gains  in  English  proficiency  as  measured  by  the 
ECLT.  The  longer  the  training  time,  the  greater  the  gains.  Oral 
proficiency  data  showed  that  six-week  students  gained  at  about  the 
same  rate  as  three-month  students  in  some  skills  but  not  in  others. 
ESL  was  found  to  have  some  positive  relationship  to  lowering  of 
attrition  rates.  Although  ESL  participants  generally  liked  their 
programs  and  teachers,  they  felt  the  need  for  more  practice  in 
conversation  skills.  Supervisors  were  generally  supportive  of  the 
programs  and  made  qualitative  distinctions  between  soldiers  scoring 
50  or  more  on  the  ECLT  and  soldiers  scoring  less  than  50. 


POLICY  IMPLICATIONS 

This  section  presents  three  main  policy  implications.  First, 
Army  ESL  programs  need  to  be  based  on  certain  key  characteristics 
suggested  by  theory  and  research.  Second,  questions  about  the  need 
for  ESL  can  be  answered  only  if  several  considerations  are  in¬ 
cluded.  Third,  the  validity  of  the  ECLT  exit  score  of  70  needs  to 
be  examined  further  for  various  Army  uses. 


Characteristics  Needed  in  Army  ESL  Programs 


The  programs  evaluated  had  mildly  positive  results.  Current 
ESL  theory  suggests  that  changing  to  a  more  functional  program 
might  produce  much  more  dramatic  results.  Information  from  the 
interviews,  observations,  and  questionnaires  also  indicates  that 
Army  ESL  programs  might  be  improved  by  greater  emphasis  on  teaching 
spoken  English;  realistic  expectations  about  what  can  be  achieved 
in  a  given  amount  of  time;  and  systematic  training  of  Army  ESL 
teachers,  most  of  whom  have  not  t"ught  ESL  before  contracting  to 
teach  for  the  Army. 
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Need  for  ESL  in  the  Army 


Four  factors  affect  the  need  for  ESL  in  the  Army: 
demographics,  ecomomics,  evaluation,  and  ethics.  Demographics  show 
that  there  is  a  large  ESL-eligible  population  in  the  Army.  The 
high  Hispanic  growth  rate  will  increase  the  number  of  recruitment- 
age  Hispanics  in  the  next  few  decades.  Some  individuals  'argue  that 
if  the  recession  continues,  the  Army  will  have  such  a  large  pool  of 
eligible  recruits  that  it  will  no  longer  have  to  induct  limited 
English  proficient  soldiers  at  all.  However,  if  the  economy 
improves  and  the  pool  of  eligible  English  speaking  recruits  shrinks 
due  to  availability  of  nonmilitary  jobs,  the  Army  may  find  once 
more  that  it  needs  high  caliber,  limited  English  proficient  sol¬ 
diers  who  can  be  trained  in  ESL.  In  that  case,  -good  ESL  training 
would  be  essential  for  the  Army.  Program  evaluation  data  also 
relate  to  the  need  for  ESL.  As  seen  in  this  report,  English 
proficiency  levels  can  be  improved  through  ESL  training  in  as  short 
a  time  as  six  weeks,  and  the  longer  the  training  time,  the  more 
improvement  can  be  made.  Evaluation  data  also  show  effects  of  ESL 
on  attrition  and  perceptions.  Finally,  the  ethical  factor  relates 
to  the  need  for  ESL.  It  may  not  be  ethical  to  deny  some  Americans, 
such  as  Puerto  Ricans,  the  opportunities  for  career  development 
that  the  Army  offers  because  those  Americans  do  not  speak  English 
as  their  native  language. 


Validity  of  ECLT  Exit  Score  of  70  for  the  Army 


The  Army  has  implicitly  accepted  DLIELC’s  criterion  of  English 
language  proficiency:  an  exit  score  of  70  on  the  ECLT.  DLIELC 
developed  this  standard  over  years  of  empirical  testing  with 
foreign  nationals.  The  Army  needs  to  investigate  whether  the 
criterion  of  70  is  relevant  to  its  own  uses.  That  score  may  be 
relevant  to  some  ESL  soldiers  and  not  others--depending  on  their 
Army  jobs,  their  ability  to  communicate  nonverbally,  and  the 
assist  'nee  network  available  through  peers  and  supervisors.  Data 
should  oe  gathered  on  the  predictive  utility  of  any  proficiency 
score  in  terms  of  performance  on  the  job.  Some  of  the  data 
presented  above  indicate  that  50  is  a  criterion  that  might  be 
relevant  for  some  Army  uses.  However,  lowering  the  proficiency 
standard  from  70  to  50  could  have  a  negative  effect  on  trainees' 
long-term  opportunities  in  the  Army  by  pushing  ESL  soldiers  out  of 
the  program  before  they  have  received  maximum  benefit.  It  must 
also  be  noted  that  any  proficiency  standard  is  somewhat  arbitrary, 
and  human  judgment  is  involved  in  the  process  of  determining  who  is 
a  competent  English  speaker  and  in  what  situations.  Empirical  data 
will  help  make  these  judgments  more  rational  in  the  Army  setting 
and  will  be  particularly  important  if  the  Army  ever  develops  an 
open  exit  program  based  on  attainment  of  a  particular  criterion 
score . 
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CONCLUSIONS 


This  report  presents  background  on  the  Army's  language  problem 
and  on  ESL  approaches.  It  also  offers  research  questions  concern¬ 
ing  three  Army  ESL  programs,  procedures  in  evaluating  the  programs, 
results  of  the  evaluation,  and  implications.  General  conclusions 
are  that  ESL  is  a  necessary  and  worthwhile  training  effort  for  the 
Army  and  that  its  effectiveness  could  be  enhanced  by  considering 
optimal  program  characteristics  and  reasonable  proficiency  stan¬ 
dards  for  ESL  participants. 


19 


REFERENCES 


Anthony,  R.  P. ,  &  Norris,  W.  E.  (1972).  Method  in  language 
teaching.  In  K.  Croft  (Ed.),  Readings  in  English  as  a  second 
language.  Cambridge,  MA:  Winthrop. 

Carroll,  J.  (1967).  The  foreign  language  attainments  of  language 
majors  in  the  senior  year;  A  survey  conducted  in  U.  S.  colleges 
and  universities.  Cambridge,  HA:  Laboratory  for  Research  in 
Instruction.  Harvard  University  Graduate  School  of  Education. 

Crandall,  J.  A.  (1979).  Adult  vocational  ESL.  Arlington,  VA : 
Center  for  Applied  Linguistics. 

Defense  Language  Institute  English  Language  Center  (DLIELC)  (1980). 
ECL  testing.  Lackland  Air  Force  Base,  TX:  DLIELC. 

Denham,  C.,  &  Lieberman,  A.  (Eds.)  (1980).  Time  to  learn. 
Washington,  DC:  National  Institute  of  Education. 

Holland,  V.  M.,  Rosenbaum,  H. ,  Stoddart  S.,  &  Redish,  J  C.  (1982). 
BSEP  I/ESL  programs.  Washington,  DC:  American  Institutes  for 
Research. 

Knowles,  M.  S.  (1978).  The  modern  practice  of  adult  education; 
Androgyny  versus  pedagogy.  New  York:  Association  Press. 

Krug,  R.  E.,  &  Wise,  L,  L.  (1982).  Analysis  of  BSEP  I  and  II  data 
files  (Research  Memorandum  82-1).  Washington,  DC:  American 
Institutes  for  Research. 

Maier,  M.  (1982).  Validation  of  selection  and  classification  tests 
in  the  Army.  Unpublished  working  paper,  U.S.  Army  Research 
Institute  for  the  Behavioral  and  Social  Sciences,  Alexandria,  VA. 

Mebane,  D.  (1980).  Renorming  of  English  Comprehension  Level  (ECL) 
examination.  Unpublished  memorandum,  Lackland  Air  Force  Base, 

TX. 

Oxford-Carpenter,  R.,  Leopold,  A.,  &  Duvall,  J.  (1982).  Fact 
sheet;  ASVAB/ECLT  study — Three-month  and  six-month  programs. 

.  Unpublished  memorandum,  U.S.  Army  Research  Institute  for  the 
Behavioral  and  Social  Sciences,  Alexandria,  7A. 

Re dish,  J.  C.  (1982).  BSEP  I/ESL  programs;  Executive  summary. 
Washington,  DC;  American  Institutes  for  Research. 

Salas,  E. ,  Kincaid,  J.  P. ,  &  Ashcroft,  N.  (1980).  An  assessment  of 
Hispanic  recruits  who  speak  English  as  a  second  language. 

Orlando,  FL;  Training  Analysis  and  Evaluation  Group. 


20 


Schumann,  J.  H.  (1978).  The  acculturation  model  for  second- 
language  acquisition*  In  R*  C.  Gingras  (Ed*)>  Se cond"*l ang 
acquisition  and  foreign  language  teaching.  Arlington,  VA  ■ 
Center  for  Applied  Linguistics. 


anguage 
VA  ■ 


Stoddart,  S.  (1982).  BT  and  AIT  follow-up  of  three-month  ESL 
course  (Research  memorandum  82-2) .  Wa shing ton ,  DC ;  American 
Tnstitutes  for  Research. 

Troike,  R.  (1983).  Can  language  be  tested?  Unpublished  draft. 
University  of  Illinois,  Urbana-Champaign ,  IL. 

U.  S.  Army  Training  and  Doctrine  Command  (TRADOC)  (1980).  TRADOC 
*evaluation--Basic  Skills  Education  Program.  Phase  I.  Fort  Monroe 
VA:  TRADOC. 


eoeiac 


